Interface interaction system

ABSTRACT

Embodiments of the present invention provide computer-implemented methods, computer program products and computer systems. Embodiments of the present invention can capture one or more interactions with an existing user interface displayed within a display screen. Embodiments of the present invention can then, in response to reaching a threshold level of interactions, dynamically generate an alternate user interface comprising one or more alternate interaction methods. Embodiments of the present invention can then overlay the alternate user interface over the existing user interface.

BACKGROUND

The present invention relates generally to the field of mobileinterfaces, and more particularly to customizing interactions withinmobile interfaces.

A user interface (UI) is the space where interactions between humans andmachines occur. The goal of this interaction is to allow effectiveoperation and control of the machine from the human end, while themachine simultaneously feeds back information that aids the operators'decision-making process. Examples of this broad concept of userinterfaces include the interactive aspects of computer operatingsystems, hand tools, heavy machinery operator controls, and processcontrols.

Generally, the goal of user interface design is to produce a userinterface which makes it user friendly to operate a machine. Userinterfaces are composed of one or more layers. For example, userinterfaces can have a human-machine interface (HMI) that interfacesmachines with physical input hardware (e.g., keyboards, mice, or gamepads, and output hardware such as computer monitors, speakers, andprinters). A device that implements an HMI is called a human interfacedevice (HID). Additional UI layers may interact with one or more humansenses, including: tactile UI (i.e., touch), visual UI (i.e., sight),auditory UI (i.e., sound), olfactory UI (i.e., smell), etc.).

SUMMARY

According to an aspect of the present invention, there is provided acomputer-implemented method. The computer implemented method comprises:capturing one or more interactions with an existing user interfacedisplayed within a display screen; in response to reaching a thresholdlevel of interactions, dynamically generating an alternate userinterface comprising one or more alternate interaction methods; andoverlaying the alternate user interface over the existing userinterface.

BRIEF DESCRIPTION OF THE DRAWINGS

Preferred embodiments of the present invention will now be described, byway of example only, with reference to the following drawings, in which:

FIG. 1 depicts a block diagram of a computing environment, in accordancewith an embodiment of the present invention;

FIG. 2 is a flowchart depicting operational steps for dynamicallygenerating an alternate user interface, in accordance with an embodimentof the present invention in accordance with an embodiment of the presentinvention;

FIGS. 3A, 3B, and 3C depict example user interfaces, in accordance withan embodiment of the present invention; and

FIG. 4 is a block diagram of an example system, in accordance with anembodiment of the present invention.

DETAILED DESCRIPTION

Embodiments of the present invention recognize certain deficiencies withmobile user interfaces. For example, when interacting with mobiledevices and touch screens, there is an issue with initiating the right,that is, correct or otherwise desired command (i.e., function, action,etc.). Single tap, Double tap, tap and hold, multi finger hold, multifinger tap, etc. are all examples of such actions that may be used on arespective area on a device screen to input or otherwise initiate aspecific action. On the other hand, using physical hardware such asmouse to move and select with a cursor in a traditional manner may notbe practical and could still present similar issues with executing thedesired command.

Embodiments of the present invention provide solutions for interactionerrors that improve user interface functionality. For example,embodiments of the present invention provide alternate interactionfunctionality that enhance user interface functionality. Specifically,embodiments of the present invention can capture repeated failedinteractions in a specific area (i.e., zone) of a touch screeninterface, generate alternate input and interaction methods, and provideand subsequently display the alternate interaction on the existing userinterface. In this manner, embodiments of the present invention canimprove user interface and technology thereof by dynamically providing(e.g., generating and deployment of) alternate input mechanisms beyonddefault interaction mechanisms as discussed in greater detail later inthis Specification.

FIG. 1 is a functional block diagram illustrating a computingenvironment, generally designated, computing environment 100, inaccordance with one embodiment of the present invention. FIG. 1 providesonly an illustration of one implementation and does not imply anylimitations with regard to the environments in which differentembodiments may be implemented. Many modifications to the depictedenvironment may be made by those skilled in the art without departingfrom the scope of the invention as recited by the claims.

Computing environment 100 includes client computing device 102 andserver computer 108, all interconnected over network 106. Clientcomputing device 102 and server computer 108 can be a standalonecomputer device, a management server, a webserver, a mobile computingdevice, or any other electronic device or computing system capable ofreceiving, sending, and processing data. In other embodiments, clientcomputing device 102 and server computer 108 can represent a servercomputing system utilizing multiple computers as a server system, suchas in a cloud computing environment. In another embodiment, clientcomputing device 102 and server computer 108 can be a laptop computer, atablet computer, a netbook computer, a personal computer (PC), a desktopcomputer, a personal digital assistance (PDA), a smart phone, or anyprogrammable electronic device capable of communicating with variouscomponents and other computing devices (not shown) within computingenvironment 100. In another embodiment, client computing device 102 andserver computer 108 each represent a computing system utilizingclustered computers and components (e.g., database server computers,application server computers, etc.) that act as a single pool ofseamless resources when accessed within computing environment 100. Insome embodiments, client computing device 102 and server computer 108are a single device. Client computing device 102 and server computer 108may include internal and external hardware components capable ofexecuting machine-readable program instructions, as depicted anddescribed in further detail with respect to FIG. 4 .

In this embodiment, client computing device 102 is a user deviceassociated with a user and includes application 104. Application 104communicates with server computer 108 to access user interactioninterface generator 110 (e.g., using TCP/IP) to access content, userinformation, and database information. Application 104 can furthercommunicate with user interaction interface generator 110 dynamicallyprovision alternate user interfaces comprising one or more alternateinput mechanisms, as discussed in greater detail in FIG. 2 .

Network 106 can be, for example, a telecommunications network, a localarea network (LAN), a wide area network (WAN), such as the Internet, ora combination of the three, and can include wired, wireless, or fiberoptic connections. Network 106 can include one or more wired and/orwireless networks that are capable of receiving and transmitting data,voice, and/or video signals, including multimedia signals that includevoice, data, and video information. In general, network 106 can be anycombination of connections and protocols that will supportcommunications among client computing device 102 and server computer108, and other computing devices (not shown) within computingenvironment 100.

Server computer 108 is a digital device that hosts user interactioninterface generator 110 and database 112. In this embodiment, userinteraction interface generator 110 resides on server computer 108. Inother embodiments, user interaction interface generator 110 can have aninstance of the program (not shown) stored locally on client computerdevice 102. In other embodiments, user interaction interface generator110 can be a standalone program or system that can be integrated in oneor more computing devices having a display screen.

User interaction interface generator 110 can dynamically provisionalternate user interfaces comprising one or more alternate inputmechanisms on an end user device. In general, user interaction interfacegenerator 110 can, with user informed consent monitor one or more userinteractions. As used herein, a user interaction can be a tactile inputfrom a respective user (e.g., as a single tap, double tap, tap and hold,multi finger hold, multi finger tap, drag and drop, pinch to zoom,etc.). Each user interaction can correspond to one or more functionalcommands. For example, functionalities provided (i.e., functional inputcommands) can include switching between applications, enlarging a screenwindow, enlarging and/or adjusting a photo. In some instances, a userinteraction can also include one or more audio and visual inputs. Forexample, user interaction interface generator 110 can access amicrophone or camera of the device to detect signs of frustrations.

For each received interaction, user interaction interface generator 110stores each received interaction and maps a corresponding location ofthe screen, and respective commands associated with the user interfacebeing displayed. In this embodiment, the location of the screen refersto a confined area within the display field of the user device capableof receiving at least tactile input (e.g., from a user). This may referto a confined area determined by pixel location or quadrant at ahardware level or application element at the software level. In thisway, user interaction interface generator 110 can track multiple userinteraction attempts and functions associated with those attempts. Insome embodiments, user interaction interface generator 110 can generatea separate graphic that can be subsequently displayed on a userinterface depicting a heat map of respective user interaction attempts.

User interaction interface generator 110 can receive information from adisplay screen having one or more tactile and pressure sensors.Information received can be one or more tactile inputs such as a singletap, double tap, tap and hold, multi finger hold, multi finger tap, dragand drop, etc. In some embodiments, user interaction interface generator110 may also have access to one or more tables that map a received inputto a respective command. For example, user interaction interfacegenerator 110 can access a table that specifies that a single tapcorresponds to an input command of “select”.

In other embodiments, user interaction interface generator 110 canreceive an indication of frustration level from the user (e.g., fromrepeated failed input). For example, in this embodiment, userinteraction interface generator 110 can access microphone informationfrom the device the user is interacting with. In this example, userinteraction interface generator 110 can detect audible grunts, sighs,and speech and interpret those sounds as expressions of frustrationusing a combination of natural language processing and topic analysis.In some embodiments, user interaction interface generator 110 canutilize biometric indicators to determine frustration levels of users(e.g., sweat, grip strength, etc.).

In this embodiment, user interaction interface generator 110 identifiesa user as being frustrated if it detects at least one of the following:more than one failed user interaction, audible or visual cues associatedwith frustration, or biometric indicators indicating frustration. Inother words, user interaction interface generator 110 detectsfrustration if any one of the above-mentioned indicators are identified.

In other embodiments, user interaction interface generator 110 cananalyze and compare a current user interaction to a user behaviorprofile to determine frustration level using a combination of machinelearning and natural language processing techniques. In response todetecting abnormal behavior, that is, behavior that deviates from abaseline behavior typically exhibited by the user (according to the userbehavior profile) user interaction interface generator 110 identifiesthe abnormal behavior as frustration. For example, user interactioninterface generator 110 can reference a user behavior profile toidentify that the user typically maximizes the user interface (e.g., fitto width of the screen) when using a certain application. However, userinteraction interface generator 110 has detected multiple failedinteractions (e.g., multiple input selections followed by cancelcommands) and identify this as abnormal behavior and thus classify thisset of interactions as frustration. Conversely, user interactioninterface generator 110 can record user behaviors such as repeatedfailed interactions and identify that as normal behavior (e.g., multipleinput selections, followed by cancel commands in an attempt to maximizethe windows). In this circumstance, user interaction interface generator110 automatically generate and display the alternate user interface thatmakes selecting the “maximize window” command easier to select.

In embodiments where a user behavior profile is not present, userinteraction interface generator 110 can continue to monitor userinteractions and build a user behavior profile. Some examples of userinteractions that can be recorded by user interaction interfacegenerator 110 can include user preferences (e.g., the user prefers tohave a larger screen when working with presentation applications, userpreference to have application windows side by side during certain hoursof the day, etc.).

In another example, user interaction interface generator 110 can measurefrustration levels based on a configured number of failed attempts. Inthis embodiment, a failed attempt is defined as a repeated interactionfollowed by a subsequent action cancellation. In this embodiment, userinteraction interface generator 110 can be configured to register afailed attempt threshold after three of the same interaction followed bysubsequent action cancellation. In other embodiments, user interactioninterface generator 110 can be configured to any optimal failed attemptthreshold.

In response to detecting a failure, user interaction interface generator110 can analyze the location of associated with the detected failure andidentify functional commands associated with the location. For example,user interaction interface generator 110 can identify the locationassociated with the failure and identify that the location providesselection functions (e.g., click level, left click, right click, hotkey)and application selection functions (e.g., drop down menu, selection,execute functionality). In another embodiment, user interactioninterface generator 110 can utilize a weighting system when there aremultiple elements nearby by examining HTML, Pixel Distance, etc. betweenuser interface event handlers on touchscreens. In this way, userinteraction interface generator 110 can catch whether a user's tactileinputs do not match with the user's intention (e.g., pressing the wrongbutton instead of hitting a red X on a browser window it keepsminimizing). In this embodiment, user interaction interface generator110 can then leverage the assigned weight values of the weighting systemto order the one or more elements based on a level of confidence (e.g.,most probable of being the input command to least).

User interaction interface generator 110 can then generate andsubsequently provision and display an alternate user interface thatpresent selectable user functions associated with the repeated failedattempts. For example, user interaction interface generator 110 cangenerate a drop down menu that is overlaid over the existing userinterface comprising a list of associated functions at a font size of1.75 times of the font size of the interaction area that was associatedwith the failed attempts. In other embodiments, user interact interfacegenerator 110 can make the generated alternate user interface fit to thewidth size of the display screen such that it replaces the existing userinterface until user interaction interface generator 110 receives aninput. In instances where there is only one functional command, userinterface generator 110 automatically initiates the input command.

In this embodiment, user interaction interface generator 110 initiates afeedback loop to capture user inputs, user sentiments, and refine itsuser interface generation. For example, any re-attempts to perform thesame action will act as a negative feedback loop to the system.Conversely, a successful action attempt will act a s positive feedbackloop to the system.

Database 112 stores received information and can be representative ofone or more databases that give permissioned access to user interactioninterface generator 110 or publicly available databases. For example,database 112 can store received source material, depicted graphics, userinteractions, user interfaces, functionalities associated with userinterfaces, interaction components, etc. In general, database 112 can beimplemented using any non-volatile storage media known in the art. Forexample, database 112 can be implemented with a tape library, opticallibrary, one or more independent hard disk drives, or multiple hard diskdrives in a redundant array of independent disk (RAID). In thisembodiment database 112 is stored on server computer 108.

FIG. 2 is a flowchart 200 depicting operational steps for dynamicallygenerating an alternate user interface, in accordance with an embodimentof the present invention.

In step 202, user interaction interface generator 110 receivesinformation. In this embodiment, user interaction interface generator110 receives information from a display screen having one or moretactile and pressure sensors (e.g., client computing device 102). Inother embodiments, where user interaction interface generator 110 isstored locally on client computing device 102, user interactioninterface generator 110 can receive information via network 106. Inother embodiments, user interaction interface generator 110 can receiveinformation from one or more other components of computing environment100.

As mentioned above information received generally refers to one or moretactile inputs that correspond to one more input command. For example,tactile information can include a single tap, double tap, tap and hold,multi finger hold, multi finger tap, drag and drop, etc. In someembodiments, user interaction interface generator 110 may also haveaccess to one or more tables that map a received input to a respectivecommand. For example, user interaction interface generator 110 canaccess a table that specifies that a single tap corresponds to an inputcommand of “select”.

In this embodiment, user interaction interface generator 110 can store amap of received tactile inputs. In this embodiment, user interactioninterface generator 110 can map corresponding locations of tactileinputs to one or more respective areas (i.e., locations) on the displayscreen and map the tactile input to a respective input command. In thisway, user interaction interface generator 110 can generate andsubsequently display a heat map of areas of the display screen receivingmultiple tactile attempts (e.g., one or more clusters of interactions).In other embodiments, user interaction interface generator 110 canaccess a mapped function list associated with a user interface ofrespective tactile inputs and respective commands those tactile inputsmap to.

In other embodiments, information can also include an indication offrustration level from the user (e.g., from repeated failed input).Information can be received via manual input from a user orautomatically. For example, user interaction interface generator 110 candetected repeated failed attempts (e.g., a sequence comprising of aninput command followed by a cancel command, repeated multiple times).User interaction interface generator 110 can also access microphoneinformation from the device the user is interacting with. In thisexample, user interaction interface generator 110 can detect audiblegrunts, sighs, and speech and interpret those sounds as expressions offrustration using a combination of natural language processing and topicanalysis. In some embodiments, user interaction interface generator 110can utilize biometric indicators to determine frustration levels ofusers (e.g., sweat, grip strength, etc.).

In step 204, user interaction interface generator 110 detects failures.In this embodiment, user interaction interface generator 110 detectsfailures after a threshold number of repeated commands are cancelled arereached or exceeded. For example, user interaction interface generator110 can detect a failure once a repeated sequence of “input command” viatactile sensor followed by a “cancel command” of that previous inputcommand. In this embodiment, the threshold number of cancelled inputcommands is three. In other words, user interaction interface generator110 detects a failure once the threshold number of repeated cancelcommands is reached or exceeded.

In another embodiment, user interface generator 110 can detect a failureas an expression of frustration. In other words, user interfacegenerator 110 can also detect the presence or absence of userfrustration. In this embodiment, user interface generator 110 can detectpresence or absence of user frustration if it detects at least one ofthe following: more than one failed user interaction, audible or visualcues associated with frustration, or biometric indicators indicatingfrustration. In other words, user interaction interface generator 110detects frustration if any one of the above-mentioned indicators areidentified.

In step 206, user interaction interface generator 110 dynamicallygenerates an alternate user interface. In this embodiment, userinteraction interface generator 110 dynamically generates an alternateuser interface by accessing a previously generated heat map of receivedtactile inputs associated with the failed attempts. User interactioninterface generator 110 can then access a mapping of input commandsassociated with the area having multiple failed attempts. Userinteraction interface generator 110 can then generate a new userinterface depicting the input commands that are associated with the areahaving failed attempts. In other words, user interaction interfacegenerator 110 determines alternate input and interaction methods for thedisplay area having multiple failed attempts and in response to reachinga threshold level of either failed interactions or frustration, generatean alternate user interface.

For example, user interaction interface generator 110 can generate adrop-down menu that is overlaid over the existing user interfacecomprising a list of associated functions at a font size of 1.75 timesof the font size of the interaction area that was associated with thefailed attempts. In other embodiments, user interact interface generator110 can make the generated alternate user interface fit to the widthsize of the display screen such that it replaces the existing userinterface until user interaction interface generator 110 receives aninput. In instances where there is only one functional command, userinterface generator 110 automatically initiates the input command.

In step 208, user interaction interface generator 110 provisions thealternate user interface. In this embodiment, user interaction interfacegenerator 110 provisions the alternate user interface on the displayscreen according to user preferences. For example, user interactioninterface generator 110 can generate and subsequently display a userinterface that resembles a pop-up window alerting a user of the screenof multiple failed attempts and a request to confirm the generation andsubsequent display of a new user interface that displays input commandsin an easier to select interface. In this embodiment, an easier toselect interface refers to an alternate user interface havingdimensional changes such as larger font size, increased paragraphspacing, etc. In other embodiments, the alternate user interface caninclude audible cues (e.g., verbal instructions to a user for inputtingcommands). In some examples, the alternate user interface may comprise adrop-down menu having selectable options fit to the width of a screen.In examples where there are multiple input commands that cannot bedisplayed with the desired font size (e.g., where only two of the fouroptions can be displayed on a single page, user interaction interfacegenerator 110 can provision a second page that the user can togglebetween with a selectable graphic icon resembling a left and rightarrow.

FIG. 3A depicts an example user interface in accordance with anembodiment of the present invention.

Specifically, FIG. 3A depicts example user interface 300. Example userinterface 300 depicts certain controls of a sound mixer. In thisexample, there are three controls, control 302, 304, and 306respectively. Each of controls 302, 304, and 306 have respective graphicicons that are selectable via tactile input and are labeled A1, A2, A3,B1, B2, mono, solo, and mute. Other graphic icons in this example alsoinclude a fader gain that can similarly be controlled by tactile inputvia a drag and drop input method.

FIG. 3B depicts an example heat map generated in accordance with anembodiment of the present invention.

FIG. 3B depicts example user interface 350. Example user interface 350is representative of example user interface 300 but with an overlaidheat map that was generated by user interaction interface generator 110.Example user interface 350 also depicts the same controls of a soundmixer (e.g., control 302, 304, and 306). In this example, userinteraction interface generator has recorded multiple user attempts andcancellation in an area of the display screen associated with control302. In response to receiving a request, user interaction interfacegenerator 110 can generate a heat map of the received user inputs. Inthis example, user interaction interface generator 110 has generatedgraphic 308 that is then subsequently overlaid over example userinterface 350 to show the area of the display screen associated withmultiple received input commands that were subsequently cancelled. Inthis manner, user interaction interface generator 110 can then pull upthe map associated with respective controls the area provided. In thisexample, user interaction interface generator 110 can use the generatedheat map graphic to identify input commands mono, solo, mute, and B2were the input commands associated with the area having multiple failedattempts.

FIG. 3C depict example of a dynamically generated user interface, inaccordance with an embodiment of the present invention.

FIG. 3C depicts example user interface 360. Example user interface 360is representative of example user interface 300 and 350 in that itdepicts the same controls control 302, 304, and 306, However, in thisexample, user interaction generator 110 has identified (e.g., using thegenerated heat map graphic) input commands mono, solo, mute, and B2 werethe input commands associated with the area having multiple failedattempts. In this example, user interaction generator 110 displays analternate user interface 310. This user interface (e.g., alternate userinterface 310) depicts the input commands mono, solo, mute, and B2 in adrop-down menu in a manner different than any of the previous exampleuser interfaces 300, 350, or 360 and is overlaid over example userinterface 360. In this example, the drop-down menu presents a “ThirdTouch Alert” (e.g., representing the threshold number of failed attemptsbeing reached) user interface that is overlaid over the originalinterface (e.g., user interface 360). This graphic presents graphicoptions associated with input commands that have been with the multiplefailed attempts (e.g., associated with the heat map, graphic 308 of FIG.3B). No other input command from example user interface 360 can beselected without closing alternate user interface 310. In thisembodiment, alternate user interface 310 can be closed by an exitfunction or by selecting and subsequently executing a command displayedin alternate user interface 310. In response to receiving user input(e.g., selection from one of the input commands), user interactioninterface generator 110 can execute program instructions associated withthe corresponding user input. For example, in response to receiving userinput such as “isolate input”, user interaction interface generator 110can execute program instructions to isolate input.

FIG. 4 depicts a block diagram of components of computing systems withincomputing environment 100 of FIG. 1 , in accordance with an embodimentof the present invention. It should be appreciated that FIG. 4 providesonly an illustration of one implementation and does not imply anylimitations with regard to the environments in which differentembodiments can be implemented. Many modifications to the depictedenvironment can be made.

The programs described herein are identified based upon the applicationfor which they are implemented in a specific embodiment of theinvention. However, it should be appreciated that any particular programnomenclature herein is used merely for convenience, and thus theinvention should not be limited to use solely in any specificapplication identified and/or implied by such nomenclature.

Computer system 400 includes communications fabric 402, which providescommunications between cache 416, memory 406, persistent storage 408,communications unit 412, and input/output (I/O) interface(s) 414.Communications fabric 402 can be implemented with any architecturedesigned for passing data and/or control information between processors(such as microprocessors, communications and network processors, etc.),system memory, peripheral devices, and any other hardware componentswithin a system. For example, communications fabric 402 can beimplemented with one or more buses or a crossbar switch.

Memory 406 and persistent storage 408 are computer readable storagemedia. In this embodiment, memory 406 includes random access memory(RAM). In general, memory 406 can include any suitable volatile ornon-volatile computer readable storage media. Cache 416 is a fast memorythat enhances the performance of computer processor(s) 404 by holdingrecently accessed data, and data near accessed data, from memory 406.

User interaction interface generator 110 (not shown) may be stored inpersistent storage 408 and in memory 406 for execution by one or more ofthe respective computer processors 404 via cache 416. In an embodiment,persistent storage 408 includes a magnetic hard disk drive.Alternatively, or in addition to a magnetic hard disk drive, persistentstorage 408 can include a solid state hard drive, a semiconductorstorage device, read-only memory (ROM), erasable programmable read-onlymemory (EPROM), flash memory, or any other computer readable storagemedia that is capable of storing program instructions or digitalinformation.

The media used by persistent storage 408 may also be removable. Forexample, a removable hard drive may be used for persistent storage 408.Other examples include optical and magnetic disks, thumb drives, andsmart cards that are inserted into a drive for transfer onto anothercomputer readable storage medium that is also part of persistent storage408.

Communications unit 412, in these examples, provides for communicationswith other data processing systems or devices. In these examples,communications unit 412 includes one or more network interface cards.Communications unit 412 may provide communications through the use ofeither or both physical and wireless communications links. Userinteraction interface generator 110 may be downloaded to persistentstorage 408 through communications unit 412.

I/O interface(s) 414 allows for input and output of data with otherdevices that may be connected to client computing device and/or servercomputer. For example, I/O interface 414 may provide a connection toexternal devices 420 such as a keyboard, keypad, a touch screen, and/orsome other suitable input device. External devices 420 can also includeportable computer readable storage media such as, for example, thumbdrives, portable optical or magnetic disks, and memory cards. Softwareand data used to practice embodiments of the present invention, e.g.,user interaction interface generator 110, can be stored on such portablecomputer readable storage media and can be loaded onto persistentstorage 408 via I/O interface(s) 414. I/O interface(s) 414 also connectto a display 422.

Display 422 provides a mechanism to display data to a user and may be,for example, a computer monitor.

The present invention may be a system, a method, and/or a computerprogram product. The computer program product may include a computerreadable storage medium (or media) having computer readable programinstructions thereon for causing a processor to carry out aspects of thepresent invention.

The computer readable storage medium can be any tangible device that canretain and store instructions for use by an instruction executiondevice. The computer readable storage medium may be, for example, but isnot limited to, an electronic storage device, a magnetic storage device,an optical storage device, an electromagnetic storage device, asemiconductor storage device, or any suitable combination of theforegoing. A non-exhaustive list of more specific examples of thecomputer readable storage medium includes the following: a portablecomputer diskette, a hard disk, a random access memory (RAM), aread-only memory (ROM), an erasable programmable read-only memory (EPROMor Flash memory), a static random access memory (SRAM), a portablecompact disc read-only memory (CD-ROM), a digital versatile disk (DVD),a memory stick, a floppy disk, a mechanically encoded device such aspunch-cards or raised structures in a groove having instructionsrecorded thereon, and any suitable combination of the foregoing. Acomputer readable storage medium, as used herein, is not to be construedas being transitory signals per se, such as radio waves or other freelypropagating electromagnetic waves, electromagnetic waves propagatingthrough a waveguide or other transmission media (e.g., light pulsespassing through a fiber-optic cable), or electrical signals transmittedthrough a wire.

Computer readable program instructions described herein can bedownloaded to respective computing/processing devices from a computerreadable storage medium or to an external computer or external storagedevice via a network, for example, the Internet, a local area network, awide area network and/or a wireless network. The network may comprisecopper transmission cables, optical transmission fibers, wirelesstransmission, routers, firewalls, switches, gateway computers and/oredge servers. A network adapter card or network interface in eachcomputing/processing device receives computer readable programinstructions from the network and forwards the computer readable programinstructions for storage in a computer readable storage medium withinthe respective computing/processing device.

Computer readable program instructions for carrying out operations ofthe present invention may be assembler instructions,instruction-set-architecture (ISA) instructions, machine instructions,machine dependent instructions, microcode, firmware instructions,state-setting data, or either source code or object code written in anycombination of one or more programming languages, including an objectoriented programming language such as Smalltalk, C++ or the like, andconventional procedural programming languages, such as the “C”programming language or similar programming languages. The computerreadable program instructions may execute entirely on the user'scomputer, partly on the user's computer, as a stand-alone softwarepackage, partly on the user's computer and partly on a remote computeror entirely on the remote computer or server. In the latter scenario,the remote computer may be connected to the user's computer through anytype of network, including a local area network (LAN) or a wide areanetwork (WAN), or the connection may be made to an external computer(for example, through the Internet using an Internet Service Provider).In some embodiments, electronic circuitry including, for example,programmable logic circuitry, field-programmable gate arrays (FPGA), orprogrammable logic arrays (PLA) may execute the computer readableprogram instructions by utilizing state information of the computerreadable program instructions to personalize the electronic circuitry,in order to perform aspects of the present invention.

Aspects of the present invention are described herein with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems), and computer program products according to embodiments of theinvention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer readable program instructions.

These computer readable program instructions may be provided to aprocessor of a general-purpose computer, a special purpose computer, orother programmable data processing apparatus to produce a machine, suchthat the instructions, which execute via the processor of the computeror other programmable data processing apparatus, create means forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks. These computer readable program instructionsmay also be stored in a computer readable storage medium that can directa computer, a programmable data processing apparatus, and/or otherdevices to function in a particular manner, such that the computerreadable storage medium having instructions stored therein comprises anarticle of manufacture including instructions which implement aspects ofthe function/act specified in the flowchart and/or block diagram blockor blocks.

The computer readable program instructions may also be loaded onto acomputer, other programmable data processing apparatus, or other deviceto cause a series of operational steps to be performed on the computer,other programmable apparatus or other device to produce a computerimplemented process, such that the instructions which execute on thecomputer, other programmable apparatus, or other device implement thefunctions/acts specified in the flowchart and/or block diagram block orblocks.

The flowchart and block diagrams in the Figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods, and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams may represent a module, a segment, or aportion of instructions, which comprises one or more executableinstructions for implementing the specified logical function(s). In somealternative implementations, the functions noted in the blocks may occurout of the order noted in the Figures. For example, two blocks shown insuccession may, in fact, be executed substantially concurrently, or theblocks may sometimes be executed in the reverse order, depending uponthe functionality involved. It will also be noted that each block of theblock diagrams and/or flowchart illustration, and combinations of blocksin the block diagrams and/or flowchart illustration, can be implementedby special purpose hardware-based systems that perform the specifiedfunctions or acts or carry out combinations of special purpose hardwareand computer instructions.

The descriptions of the various embodiments of the present inventionhave been presented for purposes of illustration but are not intended tobe exhaustive or limited to the embodiments disclosed. Manymodifications and variations will be apparent to those of ordinary skillin the art without departing from the scope and spirit of the invention.The terminology used herein was chosen to best explain the principles ofthe embodiment, the practical application or technical improvement overtechnologies found in the marketplace, or to enable others of ordinaryskill in the art to understand the embodiments disclosed herein.

What is claimed is:
 1. A computer-implemented method comprising:capturing one or more interactions with an existing user interfacedisplayed within a display screen; in response to reaching a thresholdlevel of failed interactions with the existing user interface and userbehavior exhibited after reaching the threshold level of failedinteractions, dynamically generating an alternate user interfacecomprising one or more alternate interaction methods that replaceselectable user functions associated with each of the failedinteractions that is currently displayed on the existing user interface;and overlaying the alternate user interface over the existing userinterface.
 2. The computer-implemented method of claim 1, furthercomprising: in response to receiving an input from the alternate userinterface, executing program instructions associated with the input. 3.The computer-implemented method of claim 1, further comprising: inresponse to detecting frustration of a user, detecting whether behaviorof the user is abnormal; and in response to detecting abnormal behaviorof the user, dynamically generating an alternate user interfacecomprising one or more alternate interaction methods.
 4. Thecomputer-implemented method of claim 1, further comprising: storing amap of received user interactions; and generating a heat map graphic tobe displayed on the display screen that shows a cluster of userinteractions.
 5. The computer-implemented method of claim 1, furthercomprising: initiating a feedback loop that captures user sentiments tothe alternate user interface.
 6. The computer-implemented method ofclaim 1, further comprising: utilizing a weighting system to weight oneor more elements that correspond to a respective function that aredepicted on the existing user interface; and dynamically generating analternate user interface that orders the one or more elements based onthe weighting system.
 7. The computer-implemented method of claim 1,further comprising: capturing discrepancies between a user's intentionand tactile inputs.
 8. A computer program product comprising: one ormore computer readable storage media and program instructions stored onthe one or more computer readable storage media, the programinstructions comprising: program instructions to capture one or moreinteractions with an existing user interface displayed within a displayscreen; program instructions to, in response to reaching a thresholdlevel of failed interactions with the existing user interface and userbehavior exhibited after reaching the threshold level of failedinteractions, dynamically generate an alternate user interfacecomprising one or more alternate interaction methods that replaceselectable user functions associated with each of the failedinteractions that is currently displayed on the existing user interface;and program instructions to overlay the alternate user interface overthe existing user interface.
 9. The computer program product of claim 8,wherein the program instructions stored on the one or more computerreadable storage media further comprise: program instructions to, inresponse to receiving an input from the alternate user interface,execute program instructions associated with the input.
 10. The computerprogram product of claim 8, wherein the program instructions stored onthe one or more computer readable storage media further comprise:program instructions to, in response to detecting frustration of a user,detect whether behavior of the user is abnormal; and programinstructions to in response to detecting abnormal behavior of the user,dynamically generate an alternate user interface comprising one or morealternate interaction methods.
 11. The computer program product of claim8, wherein the program instructions stored on the one or more computerreadable storage media further comprise: program instructions to store amap of received user interactions; and program instructions to generatea heat map graphic to be displayed on the display screen that shows acluster of user interactions.
 12. The computer program product of claim8, wherein the program instructions stored on the one or more computerreadable storage media further comprise: program instructions toinitiate a feedback loop that captures user sentiments to the alternateuser interface.
 13. The computer program product of claim 8, wherein theprogram instructions stored on the one or more computer readable storagemedia further comprise: program instructions to utilize a weightingsystem to weight one or more elements that correspond to a respectivefunction that are depicted on the existing user interface; and programinstructions to dynamically generate an alternate user interface thatorders the one or more elements based on the weighting system.
 14. Thecomputer program product of claim 8, wherein the program instructionsstored on the one or more computer readable storage media furthercomprise: program instructions to capture discrepancies between a user'sintention and tactile inputs.
 15. A computer system comprising: one ormore computer processors; one or more computer readable storage media;and program instructions stored on the one or more computer readablestorage media for execution by at least one of the one or more computerprocessors, the program instructions comprising: program instructions tocapture one or more interactions with an existing user interfacedisplayed within a display screen; program instructions to, in responseto reaching a threshold level of failed interactions with the existinguser interface and user behavior exhibited after reaching the thresholdlevel of failed interactions, dynamically generate an alternate userinterface comprising one or more alternate interaction methods thatreplace selectable user functions associated with each of the failedinteractions that is currently displayed on the existing user interface;and program instructions to overlay the alternate user interface overthe existing user interface.
 16. The computer system of claim 15,wherein the program instructions stored on the one or more computerreadable storage media further comprise: program instructions to, inresponse to receiving an input from the alternate user interface,execute program instructions associated with the input.
 17. The computersystem of claim 15, wherein the program instructions stored on the oneor more computer readable storage media further comprise: programinstructions to, in response to detecting frustration of a user, detectwhether behavior of the user is abnormal; and program instructions to inresponse to detecting abnormal behavior of the user, dynamicallygenerate an alternate user interface comprising one or more alternateinteraction methods.
 18. The computer system of claim 15, wherein theprogram instructions stored on the one or more computer readable storagemedia further comprise: program instructions to store a map of receiveduser interactions; and program instructions to generate a heat mapgraphic to be displayed on the display screen that shows a cluster ofuser interactions.
 19. The computer system of claim 15, wherein theprogram instructions stored on the one or more computer readable storagemedia further comprise: program instructions to initiate a feedback loopthat captures user sentiments to the alternate user interface.
 20. Thecomputer system of claim 15, wherein the program instructions stored onthe one or more computer readable storage media further comprise:program instructions to utilize a weighting system to weight one or moreelements that correspond to a respective function that are depicted onthe existing user interface; and program instructions to dynamicallygenerate an alternate user interface that orders the one or moreelements based on the weighting system.